Spectral imaging with a color wheel

ABSTRACT

A spectral imaging apparatus includes an image sensor positioned to capture images of a target, a rotating color wheel aligned with the image sensor, and a projector positioned to project overlay images onto the target. The color wheel can include one or more filter segments, each allowing light of a different range of wavelengths to pass. The color wheel can further include a blocking or opaque segment. A set of captured images of the different wavelengths can be processed to generate a false-color overlay image for projection onto the target. Processing of overlay images can be performed when the blocking or opaque segment is in front of the image sensor. The spectral imaging apparatus may be a multi-spectral imaging apparatus and made be used in the medical fields, such as for determining and indicating tissue oxygenation. Video rates can be achieved.

FIELD

This disclosure relates to imaging, and more particularly, to spectralimaging.

BACKGROUND

Multi-spectral and hyper-spectral imagers are 2D imaging devices thatcollect spectral information over a spectral range, such as discretewavelength ranges (multi-spectral) or continuous wavelength ranges(hyper-spectral). Such devices may be used to obtain spatially resolvedspectral information for applications such as agriculture and foodprocessing where spectrally and spatially resolved information can beused to assess moisture in crops and bruising in fruits. Similartechnology is also used in medical applications to determine tissueoxygen level, for example.

The typical device uses a 2D imager and optics containing a dispersingprism or grating. The device operates as a line scanner in which asample passing by the device is scanned and the incoming light isdispersed onto an imager. As the device completes the scan of theobject, an image of the object is created that is spectrally resolved.The spectrally resolved image can then be devolved into individualwavelengths allowing for identification of chemicals that contribute tothe spectral response in the image. As an example, identifying a waterspectral component in such an image enables users to then encode theimage for water content and show the chemical signature of water in theimage. This is one application of spectral imaging for crop fields.Another type of spectral imager captures a full field image for eachwavelength. In this type of design, the object is illuminated at variouswavelengths and for each wavelength, an image is captured. The capturedimage cube can then be analyzed and chemically resolved to display thechemical of interest in a multi-wavelength image. Another alternative tothe above design uses a tunable filter or a set of optical filters thatare scanned past the imager to generate the image cube for chemicalencoding.

One drawback of the above techniques is processing speed, which isgoverned by either how fast the object can move past the spectral linescanner or how quickly each wavelength can be captured in the imager.For applications where it is not possible for the object to be moving,such as a patient, the acquisition time can be very long. A chemicallyencoded image may take tens of seconds to generate, making it infeasiblefor real-time measurements.

SUMMARY

A rotating color wheel filters light from a target captured by an imagesensor. Captured images can be processed into overlay images that can beprojected back onto the target. In addition to one or more filtersegments, the color wheel can include a blocking or opaque segment andprocessing of overlay images can be performed when the blocking oropaque segment prevents capture of target light at the image sensor. Thecolor wheel can be rotated and the image processing and projection canbe performed at video rates.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings illustrate, by way of example only, embodiments of thepresent invention.

FIG. 1 is a block diagram of a spectral imaging apparatus according toembodiments of the present invention.

FIG. 2 is a block diagram of the imager.

FIG. 3 is a block diagram of the image processor.

FIG. 4 is a front view of the color wheel.

FIG. 5 is a timing diagram of the spectral imaging apparatus.

FIG. 6 is a state diagram illustrating a method of timing the capture,processing, and projecting of images according to embodiments of thepresent invention.

FIG. 7 is a schematic diagram of a set of captured images and agenerated overlay image.

FIG. 8 is a front view of a color wheel according to another embodimentof the present invention.

FIG. 9 is a front view of a color wheel according to yet anotherembodiment of the present invention.

FIG. 10 is a block diagram of a portion of an imager according toanother embodiment.

DETAILED DESCRIPTION

It is to be understood that this invention is not limited to theparticular structures, process steps, or materials disclosed herein, butis extended to equivalents thereof as would be recognized by thoseordinarily skilled in the relevant arts. It should also be understoodthat terminology employed herein is used for the purpose of describingparticular embodiments only and is not intended to be limiting.

It should also be understood that the functions, processes, and methodsdescribed in this specification may be embodied in programs stored inmemory and executable by a processor. Programs may indeed be implementedin software for execution by various types of processors. An identifiedprogram of executable code may, for instance, comprise one or morephysical or logical blocks of computer instructions, which may, forinstance, be organized as an object, procedure, class, function, orsimilar programmatic entity. Nevertheless, the executables of anidentified program need not be physically located together, but maycomprise disparate instructions stored in different locations which,when joined logically together, comprise the program and achieve thestated purpose for the program.

A program may also be implemented as a hardware circuit comprisingcustom VLSI circuits or gate arrays, off-the-shelf semiconductors suchas logic chips, transistors, or other discrete components. A program mayalso be implemented in programmable hardware devices such as fieldprogrammable gate arrays, programmable array logic, programmable logicdevices or the like.

Indeed, a program of executable code may be a single instruction, ormany instructions, and may even be distributed over several differentcode segments, among different programs, and across several memorydevices. Similarly, operational data may be identified and illustratedherein within programs, and may be embodied in any suitable form andorganized within any suitable type of data structure. The operationaldata may be collected as a single data set, or may be distributed overdifferent locations including over different storage devices, and mayexist, at least partially, merely as electronic signals on a system ornetwork. The programs may be passive or active, including agentsoperable to perform desired functions.

Reference throughout this specification to “one embodiment” or “anembodiment” means that a particular feature, structure, orcharacteristic described in connection with the embodiment is includedin at least one embodiment of the present invention. Thus, appearancesof the phrases “in one embodiment” or “in an embodiment” in variousplaces throughout this specification are not necessarily all referringto the same embodiment.

FIG. 1 illustrates a spectral imaging apparatus 10 according to anembodiment of the present invention. The spectral imaging apparatus 10may be known as a multi-spectral imaging apparatus or a hyper-spectralimaging apparatus. The spectral imaging apparatus 10 will be describedin the context of medical use, however, it should be understood that theapparatus 10 may find use in other fields, such as food processing oragriculture.

The imaging apparatus 10 includes an imager 12, an image processor 14, aprojector 16, and an optical filter 18. The image processor 14 isconnected to the imager 12 and the projector 16, and is configured toprocess image information collected by the imager 12 and output imageinformation to the projector 16. The imager 12 and projector 16 collectlight from and emit light onto a target object, such as a target tissue20 (e.g., a region of a patient's skin surface) via the filter 18. Aswill be discussed, the imaging apparatus 10 is capable of spectraltissue oxygenation measurements, measuring and processing spectralinformation at video-rate speeds, and projecting a chemically encodedoxygen map directly back onto the target tissue.

Tissue oxygen level plays a key role in managing patient wounds and insurgical procedures where tissue viability is important. In specificapplications such as burn wounds, amputations, diabetic ulcers, andcosmetic surgery, tissue oxygen level can be a direct measure of tissuehealth and healing progress. The imaging apparatus 10 can assistphysicians and other healthcare workers in assessing tissue health inreal-time or near real-time, that is, at video rates, in order toaugment or possibly replace experience and direct visual inspection. Asa result, patients can experience shorter healing cycles due to reducedchance of infection or repeating of procedures. This may also reduce theburden on the healthcare system.

The imager 12 emits light onto a region 22 of the target tissue 20 toilluminate the region 22 of target tissue 20. The region 22 is a portionof target tissue that is of interest and can include, for example, aportion of a patient's skin or another organ or portion thereof.

The filter 18 is configured to allow some light to pass while reflectingremaining light. The filter 18 may include a bandpass filter, ashortpass filter, a notch filter, a dichroic filter, or the like. Thefilter 18 is arranged to reflect the illumination light emitted by theimager 12 onto the target tissue 20 and to direct light reflected fromthe target tissue 20 along an imaging optical path 24 back to the imager12. The imaging optical path 24 extends between the target tissue 20 andthe imager 12. In this embodiment, the band pass filter 18 is tilted atabout 45 degrees with respect to the imaging optical path 24 at theimager 12.

The imager 12 captures light reflected from the target tissue 20 alongthe imaging optical path 24 via the band pass filter 18.

In this embodiment, illumination light emitted from the imager 12 alsotravels along the imaging optical path 24. However, it should beunderstood that light emitted by the imager 12 need not be entirelycoincident with light reflected to the imager 12 from the target tissue20 in order for both emitted light and incoming light to be consideredto share the imaging optical path 24. In other embodiments, illuminationlight is emitted along a different optical path.

The image processor 14 is configured to generate overlay images(frames), such as false color images, referencing light captured by theimager 12. The image processor 14 provides the overlay images to theprojector 16.

The projector 16 is arranged to project overlay images along aprojecting optical path 26 and onto the target tissue 20. The projector16 can be any suitable kind of projector 16 and may include an I/Ointerface for connecting to the image processor 14, a controller forcontrolling operations of the projector 16, a projection light source(lamp, laser, LED), and an imaging device, such as LCD light valves, adigital micro-mirror device (DMD), or similar.

The projecting optical path 26 extends from the projector 16 to thetarget tissue 20 via the filter 18. The projecting optical path 26 atleast partially overlaps the imaging optical path 24, as shown at 28. Inthis embodiment, the optical paths 24, 26 overlap between the filter 18and the target tissue 20, and do not overlap between the filter 18 andthe imager 12 or projector 16. The projecting optical path 26 and theimaging optical path 24 partially overlapping (coaxial alignment) inthis manner is advantageous in that overlay images can be projecteddirectly onto the object (e.g., tissue 20) being examined and willremain aligned irrespective of focus distance or projection angle.

The filter 18 is arranged along the projecting optical path 26 and theimaging optical path 24, and specifically where these paths intersect,so as to reflect specific wavelengths of light while allowing otherwavelengths to pass. In this embodiment, the imager 12 emits light of aset of wavelengths. (A set or subset of wavelengths can include a rangeof wavelengths or a discrete one or more wavelengths.) The filter 18 isselected to reflect at least a subset of the set of wavelengths towardsthe target tissue 20, with light of the remaining wavelengths passingthrough the filter 18 and out of the system. Light of the subset ofwavelengths strikes the target tissue 20 and a further subset ofwavelengths is reflected from the target tissue 20 back towards thefilter 18, which reflects most or all of the further subset of light tothe imager 12. This further subset of wavelengths of light isrepresentative of a characteristic of the target tissue 20 beingmeasured, such as tissue oxygenation, and is processed by the imageprocessor 14 into at least one overlay image that is projected by theprojector 16. The overlay image can be generated with reference to theproperties of the filter 18, so that most or all of the light of theoverlay image projected by the projector 16 passes through the filter 18and onto the target tissue 20. Irrespective of the wavelengthcomposition of the overlay image, the filter 18 reflects away lightemitted by the projector 16 that would be erroneously captured by theimager 12. Naturally, the color composition of the overlay images can beselected so that all information of interest is projected onto thetarget tissue 20. However, it is advantageous that the filter 18 ensuresthat very little, if any, light emitted by the projector 16 enters thesystem at the imager 12, since such light may introduce error.

Referring to FIG. 2, which shows components of an embodiment of theimager 12, a color wheel 40 is used in the capture of light of thefurther subset of wavelengths that is reflected from target tissue 20.

The imager 12 includes the color wheel 40 positioned between an imagesensor 42 and an imaging lens 44, which is positioned to capture lightreflected from the target tissue 20. In this embodiment, the imager 12further includes a series of relay lenses 46 (illustrated schematically)positioned between the color wheel 40 and the image sensor 42. Inanother embodiment, such as that shown in FIG. 10, the color wheel 40 islocated in front of the image sensor 42 and the relay lenses 46 arelocated behind the imaging lens 44. That is, the relay lenses 46 arelocated between the color wheel 40 and the imaging lens 44. The colorwheel 40 may be placed where the diameter of the light rays is smallestas this advantageously allows the color wheel 40 to be made as small aspossible. To achieve this, the relay lenses 46 can be selected andarranged to provide an image beam that is compressed.

The image sensor 42 can be selected as a high-speed and high-sensitivityCMOS sensor, such as those commercially available from ON Semiconductorof Phoenix, Ariz. Examples of such sensors include those available underproduct numbers LUPA300 and LUPA1300. These are merely illustrativeexamples, and other image sensors having suitable frame rates andsensitivities can be used.

The lenses 44, 46 can be selected for the particular geometry andoptical characteristics desired.

The color wheel 40 is aligned with the image sensor 42 on the imagingoptical path 24. The color wheel 40 is configured to rotate at apredetermined frequency, such as at least about 15 Hz to attainminimally acceptable video frame rates. The color wheel 40 includes atleast one filter segment and at least one light blocking (e.g., opaque)segment, as will be discussed below.

The imager 12 further includes, in this embodiment, a motor 50 connectedto the color wheel 40 to rotate the color wheel 40, a controller 52connected to the motor 50, an input/output (I/O) circuit 54 connected tothe controller 52 and to the image sensor 42, and an illumination devicethat includes a illumination source 56 and an illumination lens 58arranged to illuminate the target tissue 20. The imager 12 can furtherinclude a housing (not shown) to contain its components.

The motor 50 has a shaft that connects to the center of the color wheel40 to rotate the color wheel 40. The motor 50 can be any suitable kindof motor.

The controller 52 includes a processor or similar device configured tocontrol operation of the motor 50. The controller 52 can referenceoutput of the image sensor 42 to determine which segment of the colorwheel 40 is currently aligned with the imaging optical path 24 and tomaintain a predetermined constant speed of rotation for the color wheel40. That is, when output of the image sensor 42 is above a thresholdamount (e.g., threshold intensity), then the controller 52 determinesthat a filter segment is currently in front of the image sensor 42.Conversely, when output of the image sensor 42 is below the thresholdamount, then the controller 52 determines that the blocking segment iscurrently in front of the image sensor 42. Such determinations can bemade periodically to measure and control the rotational frequency of thecolor wheel 40. As such, the image sensor can be considered to also be arotational position sensor for the color wheel 40.

In other embodiments, the motor 50 can include a rotational positionsensor 60 that provides rotational position measurements of the motor 50and thus the color wheel 40 to the controller 52. The controller 52 canbe configured to reference signals received from the rotational positionsensor 60 to maintain a predetermined constant speed of rotation for thecolor wheel 40. The rotational position sensor 60 can also be referencedby the controller 52 to determine which segment of the color wheel 40 iscurrently aligned with the imaging optical path 24.

The controller 52 is further configured to control illumination of theillumination source 56, which can include a while-light LED, combinationof wavelength matched LEDs, a Xenon (Xe) lamp or wavelength matchedlasers to emit light onto the target tissue 20. The controller 52 cansynchronize light emissions from the light source with reference to therotational position of the color wheel 40. In this embodiment, thecontroller 52 turns off the light source whenever a blocking segment ofthe color wheel 40 blocks light on the imaging optical path 24, whichcan advantageously save power. This also has the benefit of projectingimages onto a non-illuminated or lesser-illuminated surface so as toimprove image quality.

The controller 52 is further configured to control the I/O circuit 54 tocapture images whenever a filter segment of the color wheel 40 isaligned with the imaging optical path 24. In some embodiments, thecontroller 52 and I/O circuit 54 are parts of the same integratedcircuit.

The I/O circuit 54 outputs images captured by the image sensor 42 to theimage processor 14. The I/O circuit 54 can be any suitable kind ofinput/output interface.

FIG. 3 illustrates an embodiment of the image processor 14. The imageprocessor 14 includes an I/O circuit 70, a buffer 72 connected to theI/O circuit 70, a processor 74 connected to the I/O circuit 70 and thebuffer 72, and memory 76 connected to the processor 74.

The I/O circuit 70 receives captured images from the imager 12(specifically, the I/O circuit 54) and outputs overlay images to theprojector 16. The I/O circuit 70 can be any suitable kind ofinput/output interface. In some embodiments, the I/O circuit 70 can bemerged with the I/O circuit 54 of the imager 12, particularly when theimager 12 and the image processor 14 are provided together in the samehousing.

The buffer 72 receives captured images from the I/O circuit 70 andbuffers the captured images for the processor 74. In some embodiments,output from the image sensor 42 is only buffered when the rotationalposition sensor (e.g., the image sensor 42, a separate sensor 60, etc)indicates that the image sensor 42 is capturing suitable images. Thebuffer 72 can be any suitable kind of buffer memory and may include adata buffer, a framebuffer, or similar.

The processor 74 is configured by, for example, a program to processcaptured images in the buffer 72 into overlay images, as will bediscussed in detail below. The processor 74 can be further configured toprovide overlay images to the projector 16 for projection whenever ablocking segment of the color wheel 40 blocks the imaging optical path24 by, for example, referencing the rotational position of the colorwheel 40. The processor 74 can be any suitable kind of processor 74capable of processing images of the selected resolution in the amount oftime defined by the frequency of rotation of the color wheel 40.

The memory 76 can include any combination of short-term and long-termstorage such as RAM, ROM, cache memory, flash memory, a hard-drive, andthe like. The memory 76 stores programs that are executed by theprocessor 74 and can further provide working storage space forprocessing of images.

Operation of the medical imaging apparatus 10 will now be discussed inmore detail with reference to FIGS. 4 and 5, which respectivelyillustrate an embodiment of the color wheel 40 and a related timingdiagram.

The color wheel 40 includes at least one filter segment 80-88 and atleast one light blocking (e.g., opaque) segment 90. In the exampleillustrated, five filter segments 80-88 are provided and a singleblocking segment 90 is provided. The filter segments are the same sizeand together span 180 degrees of the color wheel 40, with the blockingsegment spanning the remaining contiguous 180 degrees.

The filter segments 80-88 are configured to allow light of differentwavelengths to pass and to block light of other wavelengths. The filtersegments 80-88 can include narrow bandpass filters chosen to matchabsorption bands of oxygenated hemoglobin. In this example, each of thefilter segments 80-88 has a different center wavelength. Two or moreabsorption bands can be used to compute the concentration of oxygenatedhemoglobin. Generally, more absorption bands (filter segments) provideincreased accuracy.

The blocking segment 90 is configured to block light from the imagesensor 42 (FIG. 2) at times when the projector 16 (FIG. 1) isprojecting. The blocking segment 90 can include black, opaque material.

The relative sizes of the filter segments 80-88 and blocking segment 90can be selected to determine the amount of time available for imagecapture and image processing/projecting. In this example, the blockingsegment 90 is selected to occupy 50% of the color wheel 40, therebyallowing the filter segments 80-88 to occupy the remaining 50% of thecolor wheel 40. Since the color wheel 40 rotates at a constantfrequency, the image capture time and image processing/projecting timeare approximately equal. The filter segments 80-88 can all have the samesize, as in the illustrated example, or can have different sizes therebyallowing different times for capture of different wavelengths.

The absorption spectra for oxygenated and deoxygenated hemoglobin tendto be in the visible and near-infrared spectrums. The color wheel filtersegments and filter 18 (FIG. 1) can be selected and configuredaccordingly.

In some embodiments, the filter 18 is a shortpass filter selected totransmit wavelengths of about 450 nm to about 650 nm and reflectwavelengths of greater than about 650 nm, up to about 850 nm.Accordingly, visible light leaves the system while near-infrared lightis reflected between the imager 12 and target 20. The color wheel filtersegments can have center wavelengths selected within the 650 to 850 nmrange according to known wavelengths of response for oxygenated anddeoxygenated hemoglobin, so as to enhance differences in oxygenated anddeoxygenated hemoglobin. Alternatively, the color wheel filter segmentscan be selected to have center wavelengths that are evenly spaced withinthe 650 to 850 nm range. The projector 16 can thus use the visible lightspectrum of about 450 nm to about 650 nm to project clear and crispimages. In these embodiments, use of near-infrared light for imagingallows for penetration into the tissue 20 to provide for a bulkassessment of tissue oxygenation.

In other embodiments, the filter 18 is a notch filter selected toreflect wavelengths between about 500 to 600 nm and allow otherwavelengths of light to pass and leave the system. Accordingly, thisrange of visible light is reflected between the imager 12 and target 20.The color wheel filter segments can have center wavelengths selectedwithin the 500 to 600 nm range according to known wavelengths ofresponse for oxygenated and deoxygenated hemoglobin, so as to enhancedifferences in oxygenated and deoxygenated hemoglobin. Alternatively,the color wheel filter segments can be selected to have centerwavelengths that are evenly spaced within the 500 to 600 nm range. Theprojector 16 uses visible light wavelengths outside the 500 to 600 nmrange for projection. In these embodiments, use of visible light forimaging allows for a surface assessment of tissue oxygenation, asvisible light does not substantially penetrate tissue.

The spectral imaging apparatus 10 can be made modular, so as to readilyprovide for bulk and/or surface tissue analysis. The color wheel 40 andfilter 18 can be configured as modules, so that they can be selected andinstalled at the time of manufacture. The color wheel 40 and filter 18can further be removable and replaceable, so that the apparatus 10 canbe reconfigured after being put into service. The image processor 14 canbe configured to support both bulk and surface tissue analysis, or canbe configured to be updatable to support bulk or surface tissue analysisdepending on the selection of the color wheel 40 and filter 18.

FIG. 5 illustrates the different times during which capture andprocessing/projecting is performed for the example color wheel 40 ofFIG. 4. Half of the period for one frame, that is, half of 1/15 seconds(i.e., 0.033 s), is dedicated to capturing images of the fivewavelengths defined by the filter segments 80-88. During the other halfof the period, when the blocking segment 90 blocks the capture of light,a spectral overlay image (frame) to encode for oxygen is processed and apreviously processed overlay image is displayed. Because suitableprocessing time is required to encode oxygen concentration informationinto overlay images, captured images are buffered and projected overlayimages lag by at least one frame, so that each displayed overlay imageis at least one frame behind the current image being processed. In thisexample, since separate images for the five wavelengths are capturedwithin 0.033 seconds, the image sensor 42 (FIG. 2) is selected to havean acquisition speed of 150 frames per second. The waveform 100illustrated represents, in some embodiments, the output of therotational position sensor 60 or the comparison of the output of theimage sensor 42 with the threshold output amount, where positive andnegative edge triggering indicates to the processor 74 (FIG. 3) what isto be performed at a given time.

As the speed of rotation of the color wheel 40 is increased, sets ofimages of the different wavelengths provided by the filter segments80-88 are captured and processed into overlay images, such that overlayimages form frames of a real-time video that is projected onto theobject (e.g., tissue 20) at video rates, such as 15 or more frames persecond.

FIG. 6 illustrates a state diagram for a method 110 of timing thecapture, processing, and projecting of images according to an embodimentof the present invention. The method 110 can be used with the spectralmedical imaging apparatus 10, and specifically, can be programmed to beperformed by the processor 74 (FIG. 3) of the image processor 14.

At 112, images are captured by the image sensor 42 when the orientationof the color wheel 40 causes light provided to the image sensor 42 to befiltered. As several filter segments 80-88 are used, a set of multiplecaptured images 118 of different wavelength bands are captured duringthe same cycle, i, of the color wheel 40 for use in generating oneoverlay image 119, as shown in FIG. 7.

At 114, when the blocking segment 90 blocks light from being captured bythe image sensor 40, the multiple images of different wavelength bandsthat were captured during the same cycle, i, of the color wheel 40 areprocessed to generate an overlay image. At least partiallycontemporaneously with such processing of captured images, a previouslygenerated overlay image of a previous cycle, i−1, is projected onto thetarget tissue. While the projected image lags by one frame, it isadvantageous that the video-rate nature of overlay image projectionmakes this lag imperceptible.

When a filter segment is next detected, the cycle index, i, is advanced,at 116. The cycle index, i, can be used to store images in the memory 76(FIG. 3) and may be used by the processor 74 to identify a particularoverlay image or set of captured images.

The filter segments of the color wheel 40 can be selected so that eachof the spectrally resolved captured images represents an image showingan absorption pattern of both oxygenated and deoxygenated hemoglobin ata particular wavelength. To compute the relative concentrations of each,a reference spectrally resolved image which takes into account sensorresponse, illumination profile on the tissue, and any backgroundlighting can be obtained. The reference image is measured using theapparatus 10 on a sample of Spectralon, which is a standard materialavailable from LabSphere with nearly 100% lambertian distribution and99.9% reflectivity. In addition, to the reference image, the dark levelintensity at the image sensor 42 (FIG. 2) can also be measured to removeany dark noise resident on the CMOS device and elsewhere in the system.The reference image and the dark level image can be measured prior toeach measurement taken of the tissue 20 (FIG. 1) or other object.Alternatively, a reference image and dark level image can be factory setand stored in memory 76 (FIG. 3) for later computation.

To compute the oxygen concentration, the processor 74 (FIG. 3) can ratiothe measured absorption intensity, reference spectra, and dark levelintensity to compute a reflectance ratio, R(λ) for each of the capturedimages at the respective wavelength, λ:

${R(\lambda)} = \frac{I_{o} - I_{b}}{I_{m} - I_{b}}$

Here I_(o) is the reference image, I_(b) is the dark level image, andI_(m), is the measured patient image all at wavelength, λ.

The reflectance ratio can be related to the absorption coefficient bythe Beer-Lambert equation and can be expressed as a linear combinationof the contribution of oxygenated and deoxygenated hemoglobin to themeasured absorption.

ln(R(λ_(i)))=(ε_(HbO2)(λ_(i))C _(HbO2)+ε_(Hb)(λ_(i))C _(Hb))L

where, ε is the molar extinction coefficient for the combination ofoxygenated and deoxygenated hemoglobin at each wavelength, C is theconcentration of oxygenated and deoxygenated hemoglobin, and L is aconstant representing an absorption path length. In the example colorwheel 40 having five filter segments 80-88, the above equation resultsin a linear set of five equations, one for each measured wavelength. Theequations are solved by a least squares fit of the oxygenated anddeoxygenated hemoglobin concentrations. The processor 74 can performthis pixel by pixel. The percent oxygenation is then calculated from:

$S_{O\; 2} = {\frac{C_{{HbO}\; 2}}{C_{{HbO}\; 2} + C_{Hb}} \times 100}$

and a resulting overlay image is generated by the processor 74 with eachpixel representing a percent oxygen level in the image. Each pixel canthen be color encoded to visually represent the oxygen concentrationover the entire overlay image. Suitable colors for the overlay image canbe selected based on the target (e.g., patient's skin) and ambientlighting conditions (e.g., hospital indoor lighting).

The above calculations can be stored in the memory 76 of the imageprocessor 14 (FIG. 3) as instructions executable by the processor 74.

FIG. 8 shows a color wheel 120 according to another embodiment. Thecolor wheel 120 can be used with any of the apparatuses and methodsdescribed herein. The color wheel 120 includes two filter segments 122,124 selected for different wavelengths and one blocking segment 126. Theblocking segment 126 occupies less than 180 degrees of the color wheel120. In another embodiment, the blocking segment 126 occupies more than180 degrees of the color wheel 120.

FIG. 9 shows a color wheel 130 according to another embodiment. Thecolor wheel 130 can be used with any of the apparatuses and methodsdescribed herein. The color wheel 130 includes two filter segments 132,134 selected for different wavelengths and one blocking segment 134. Inthis embodiment, the filter segments 132, 134 are not the same size. Inother embodiments, other quantities of different-sized filter segmentsare provided. When the color wheel rotates at a constant speed, filtersegments of larger sizes increase the time for light collection and canthus be useful for improving collection efficiency at wavelengths wherethe available signal is low. After capture and during processing (e.g.,at 114 of FIG. 6), signal correction can be performed to scale thesignal back down to normalize signal intensity ratios. This may improvethe signal-to-noise ratio at wavelengths where the signal is weak. Forwavelengths where the signal is expected to be relatively stronger,filter segments can be made smaller.

FIG. 10 shows a portion of an imager 140 according to anotherembodiment. The imager 140 is similar to the imager 12 of FIG. 2 andonly differences will be discussed in detail. The imager 140 can be usedwith any of the apparatuses and methods described herein. The imager 140includes one or more relay lenses 142 (illustrated schematically)located between the imaging lens 44 and the color wheel 40, which islocated directly in front of the image sensor 42. As shown, the relaylenses are configured to reduce the beam width at the color wheel 40 soas to reduce the size of the color wheel 40. In still other embodiments,one or more relay lenses are located between the image sensor 42 and thecolor wheel 40 and one or more relay lenses are located between thecolor wheel 40 and the imaging lens 44.

As can be understood from the above, the apparatus and methods providedby the present invention advantageously allow for real-time videoprojection of false-color overlay images onto the target being examined.Although the main example is described with respect to tissueoxygenation in the medical arts, the present invention may find use inother fields where real-time overlay images are needed.

While the foregoing provides certain non-limiting example embodiments,it should be understood that combinations, subsets, and variations ofthe foregoing are contemplated. The monopoly sought is defined by theclaims.

What is claimed is:
 1. A spectral medical imaging apparatus comprising:an illumination device arranged to illuminate a target tissue; an imagesensor arranged to capture light reflected from the target tissue alongan imaging optical path; a color wheel aligned with the image sensor onthe imaging optical path, the color wheel configured to rotate at apredetermined frequency, the color wheel comprising at least one filtersegment and at least one blocking segment; a projector arranged toproject overlay images along a projecting optical path and onto thetarget tissue, the projecting optical path at least partiallyoverlapping the imaging optical path; and a processor connected to theimage sensor and the projector, the processor configured to generate theoverlay images from light captured by the image sensor and to providethe overlay images to the projector for projection when the blockingsegment blocks the imaging optical path.
 2. The apparatus of claim 1,wherein the processor is further configured to, during a current cycleof the color wheel, encode a first image based on light captured by theimage sensor during the current cycle and control the projector toproject a first image encoded during a previous cycle of the color wheelthat occurred earlier than the current cycle.
 3. The apparatus of claim1, wherein the filter segment is selected to match an absorption band ofoxygenated hemoglobin.
 4. The apparatus of claim 1, wherein the colorwheel comprises two or more filter segments.
 5. The apparatus of claim4, wherein the filter segments are selected to have differentwavelengths to match different absorption bands of oxygenatedhemoglobin.
 6. The apparatus of claim 1, wherein the color wheelcomprises a single blocking segment.
 7. The apparatus of claim 1,further comprising a filter arranged along the projecting optical pathand the imaging optical path, the filter configured to reflectwavelengths of light that the filter segment of the color wheeltransmits.
 8. The apparatus of claim 7, wherein the filter is located atan intersection of the projecting optical path and the imaging opticalpath and the filter is oriented at about 45 degrees to both theprojecting optical path and the imaging optical path.
 9. The apparatusof claim 8, wherein the projecting optical path overlaps the imagingoptical path between the filter and the target tissue.
 10. The apparatusof claim 9, wherein the projecting optical path does not overlap theimaging optical path between the filter and the image sensor.
 11. Theapparatus of claim 1, further comprising an imaging lens, the colorwheel being positioned between the image sensor and the imaging lens.12. The apparatus of claim 11, further comprising at least one relaylens positioned between the color wheel and the image sensor.
 13. Theapparatus of claim 11, further comprising at least one relay lenspositioned between the color wheel and the imaging lens.
 14. Theapparatus of claim 1, wherein the predetermined frequency of rotation ofthe color wheel is at least about 15 Hz.
 15. A method comprising:capturing images of a target tissue, the images being of differentwavelength bands defined by a color wheel; processing captured images togenerate overlay images; projecting the overlay images onto the targettissue; and timing the capturing, processing, and projecting accordingto the rotation of the color wheel to perform the projecting of apreviously generated overlay image at least partially contemporaneouslywith the processing of current captured images.
 16. The method of claim15, wherein the different wavelength bands are selected to matchdifferent absorption bands of oxygenated hemoglobin.
 17. The method ofclaim 15, wherein the color wheel comprises a blocking segment.
 18. Themethod of claim 15, wherein the capturing and projecting are performedat least partially along a same optical path.
 19. The method of claim15, comprising rotating the color wheel at least about 15 Hz.
 20. Aspectral imaging apparatus comprising: a processor; memory connected tothe processor; an image sensor connected to the processor, the imagesensor positioned to capture images of a target; a projector connectedto the processor, the projector positioned to project overlay imagesonto the target; and a color wheel aligned with the image sensor, thecolor wheel including at least two filter segments of differentwavelengths, the color wheel configured to rotate at a predeterminedfrequency; the processor configured to process a set of images of thedifferent wavelengths captured by the image sensor to generate afalse-color overlay image, and provide the overlay image to theprojector for projection onto the target.
 21. The apparatus of claim 20,wherein the processor is configured to process a set of images of thedifferent wavelengths to generate a false-color overlay image andprovide the overlay image to the projector according to predeterminedvideo-rate frequency of rotation of the color wheel.